Multi-Modal Human-Machine Communication for Instructing Robot Grasping Tasks

A major challenge for the realization of intelligent robots is to supply them with cognitive abilities in order to allow ordinary users to program them easily and intuitively. One way of such programming is teaching work tasks by interactive demonstration. To make this effective and convenient for the user, the machine must be capable to establish a common focus of attention and be able to use and integrate spoken instructions, visual perceptions, and non-verbal clues like gestural commands. We report progress in building a hybrid architecture that combines statistical methods, neural networks, and finite state machines into an integrated system for instructing grasping tasks by man-machine interaction. The system combines the GRAVIS-robot for visual attention and gestural instruction with an intelligent interface for speech recognition and linguistic interpretation, and an modality fusion module to allow multi-modal task-oriented man-machine communication with respect to dextrous robot manipulation of objects.Comment: 7 pages, 8 figure

Neoadjuvant continuous infusion of weekly 5-fluorouracil and escalating doses of oxaliplatin plus concurrent radiation in locally advanced oesophageal squamous cell carcinoma: results of a phase I/II trial

Oxaliplatin and 5-fluorouracil have a significant activity in locally advanced oesophageal squamous cell cancer (OSCC). However, their optimal dosage and efficacy when combined with concurrent radiotherapy as neoadjuvant treatment are unknown. This non-randomised, phase I/II study aimed to define the maximum tolerated dose (MTD) and assessed the histopathological tumour response rate to neoadjuvant oxaliplatin in weekly escalating doses (40, 45, 50 mg m−2) and continuous infusional 5-fluorouracil (CI-5FU; 225 mg m−2) plus concurrent radiotherapy. Patients had resectable OSCC. Resection was scheduled for 4–6 weeks after chemoradiotherapy. During phase I (dose escalation; n=19), weekly oxaliplatin 45 mg m−2 plus CI-5FU 225 mg m−2 was established as the MTD and was the recommended dosage for phase II. Oesophageal mucositis was the dose-limiting toxicity at higher doses. During phase II, histopathological responses (<10% residual tumour cells within the specimen) were observed in 10 of 16 patients (63%; 95% confidence interval: 39–82%). Overall, 16 of the 25 patients (64%) who underwent resection had a histopathological response; tumour-free resection (R0) was achieved in 80%. Neoadjuvant weekly oxaliplatin 45 mg m−2 plus CI-5FU 225 mg m−2 with concurrent radiotherapy provides promising histological response rates and R0 resection rates in locally advanced OSCC

The role of recent real-world versus future events in the comprehension of referentially ambiguous sentences: Evidence from eye tracking

Abashidze D, Knoeferle P, Carminati MN, Essig K. The role of recent real-world versus future events in the comprehension of referentially ambiguous sentences: Evidence from eye tracking. In: Kokinov B, Karmiloff-Smith A, Nersessian N, eds. European Perspectives on Cognitive Science: Proceedings of the European Conference on Cognitive Science. New Bulgarian University; 2011: 1-6.Eye tracking results suggest that people prefer to rely upon recently depicted (vs. possible future) clipart events during spoken sentence comprehension: When the verb in NP1- VERB-ADV-NP2 sentences was referentially ambiguous between a recently performed action and an equally plausible future action, people more frequently inspected the recently acted upon object during the verb (vs. an equally plausible object that hadn't yet been acted upon, Knoeferle & Crocker, 2007). Two real-world eye-tracking studies examined to which extent these findings generalize. Experiment 1 replicated them with real-world actions: A recently acted upon object (pancakes) was inspected more often than the equally plausible target of a future action (strawberries) during and shortly after participants had heard the verb. Experiment 2 had the same design but (unlike Experiment 1) the experimenter performed an action both before (e.g., sugaring the pancakes) and after (e.g., sugaring the strawberries) each trial. Each participant thus saw the “future” and the “past” action of the sentence 50 / 50. The gaze pattern in Experiment 2 was not distinguishable from that in Experiment 1. These findings suggest (a) recent real-world actions can rapidly influence comprehension (as indexed by gaze to relevant objects), and (b) people prefer to direct their attention towards a recent action target (vs. an object that will be acted upon in the near future), even when past and future actions occurred with equal frequency. Recent within-experiment frequency apparently didn’t modulate the preference to inspect the target of a past event (vs. the target of a future event)